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Abstract — Generalized Tanner graphs have been implicitly 
studied by a number of authors under the rubric of generalized 
parity-check matrices. This work considers the conditioning of 
binary hidden variables in such models in order to break all 
cycles and thus derive optimal soft-in soft-out (SISO) decoding 
algorithms. Conditionally cycle-free generalized Tanner graphs 
are shown to imply optimal SISO decoding algorithms for the first 
order Reed-Muller codes and their duals - the extended Hamming 
codes - which are substantially less complex than conventional 
bit-level trellis decoding. The study of low-complexity optimal 
SISO decoding algorithms for the family of extended Hamming 
codes is practically motivated. Specifically, it is shown that 
exended Hamming codes offer an attractive alternative to high- 
rate convolutional codes in terms of both performance and 
complexity for use in very high-rate, very low-floor, serially 
concatenated coding schemes. 

I. Introduction 

Very high-rate codes are of great interest for a number of 
practical applications including data storage and high-speed 
fiber links. The design of modern error-correcting codes which 
simultaneously have very low error floors (e.g. < 10^^" bit 
error rate) and very high rates (e.g. > 0.9) is a particularly 
challenging problem of practical interest. Due to the inherent 
difficulty of simulating the performance of codes in the very 
low floor region, the design of such codes relies on the 
principles of uniform interleaver analysis (cf. [1] and the 
references therein). To review, on the additive white Gaussian 
noise (AWGN) channel with binary antipodal signaling, the 
bit error rate (Pf,) and codeword error rate (Pew) of a large 
class of modern codes vary asymptotically with block size as 



Pr. 



(1) 



where N is the interleaver size and amax is the maximum 
exponent of N in an asymptotic union bound approximation. 
Note that a,nax depends on both the specific code construction 
and constituent codes used. If the bit (codeword) error rate 
decays with N, then the code is said to exhibit interleaver 
gain in bit (codeword) error rate. Designs for codes with low 
floors require interleaver gain in both bit and codeword error 



Serially concatenated code (SCC) constructions (i.e. codes 
composed of an inner code and outer code separated by a 
random-like interleaver) are well-suited to low-floor design 
because, provided the inner code is recursive, the maximum 
exponent of Equation [T] is [1] 



(2) 



rates thus require a„ 



< -2. 



where c?min,o is the minimum distance of the outer code. Since 
the rate of an SCC is equal to the product of the rates of its 
constituent codes, the design of a high-rate, low-floor SCC 
requires a high-rate outer code satisfying dmin.o > 3. However, 
it is very challenging to find such outer codes for which there 
exist low-complexity optimal SISO decoding algorithms. To 
this end, Graell i Amat et al. introduced a class of high-rate 
convolutional codes with optimal SISO decoding algorithms 
of moderate complexity based on their dual codes [2], [3]. 

An alternative approach to the design of high-rate low-floor 
codes are the systematic with serially concatenated parity (S- 
SCP) codes proposed in [4], of which Jin et al.'s generalized 
repeat accumulate (GRA) codes [5] are an example. The S- 
SCP code structure can be viewed as a systematic code with a 
parity generating concatenated system that resembles an SCC. 
It was demonstrated in [4] that S-SCP codes have the same 
maximum error exponent and design rules as SCCs: codes 
constructed with a parity generating system composed of a 
recursive inner parity generator and an outer code satisfying 
dmin.o > 3 achieve interleaver gain in both bit and codeword 
error rates. In contrast to SCCs, good S-SCPs can be con- 
structed with inner parity generators that have rate greater than 
1 so that the rate of outer code can be lower than the overall 
code rate thus alleviating the aforementioned challenge of 
finding high-rate, dmin.o > 3 outer codes with low-complexity 
SISO decoding algorithms. 

The design of good high-rate, low-floor codes has thus 
been largely solved for the AWGN channel. However, the S- 
SCP design philosophy is not directly applicable to the large 
class of systems which have recursive channels. The term 
recursive channel is introduced to describe systems in which 



the aggregate of the modulation and (possible) precoding with 
the channel is recursive. Continuous phase modulations over 
AWGN and fading channels as well as certain models for 
preceded magnetic recoding channels (e.g. EPR4 [6]) are 
examples of recursive channels. 

In light of the above discussion, high-rate, low-floor codes 
for use in systems with recursive channels can be constructed 
via the serial concatenation of a high-rate code and the channel 
(where the recursive channel is treated like an inner code). 
The outer high-rate code can be a modern (e.g. SCC or S- 
SCP) code or a classical (e.g. algebraic or convolutional) 
code. Classical outer codes are more attractive, however, for 
application in practical systems. Specifically, the use of a 
classical outer code for which there exists a simple non- 
iterative SISO decoding algorithm offers reductions in decod- 
ing complexity, decoding latency, and required memory with 
respect to a modern, iteratively decoded outer code. As with 
the design of SCCs for the AWGN channel, the design of such 
codes requires a high-rate, c?min > 3 outer classical code for 
which there exists a low-complexity optimal SISO decoding 
algorithm. 

To this end, the present work introduces novel low- 
complexity SISO decoding algorithms for the family of first- 
order Reed-Muller codes (and hence their duals, the extended 
Hamming codes [7]) in Section |lll] based on variable con- 
ditioning in generalized Tanner graphs (GTGs), which are 
defined in Section HI] It is shown in Section |IV] that extended 
Hamming codes offer an attractive alternative to the high-rate 
convolutional codes studied in [2], [3] for use as outer codes in 
serial concatenation with recursive inner channels. Concluding 
remarks are given in Section [V] 

II. Conditionally Cycle-Free GTGs 
A. Generalized Code Extensions 

Let C be an [n, fc, d] binary linear block code and let 



IC{1,2, 



(3) 



be some subset of the coordinate index set of C. A generalized 
extension of C, C, is formed by adding a parity-check on 
the bits corresponding to the index set X to C (i.e. a partial 
parity-check). Note that if I = {l,2,...,n} then C is 
simply a classically defined extended code [7]. The generalized 
extended code C has length n + 1, dimension k and minimum 
distance either d or d + 1 depending on the choice of I. More 
generally, a degree-g generalized extension of C is formed by 
adding g partial parity bits to C. Note that the j*'' partial parity 
bit, Cn+j, in such an extension forms a partial parity on any 
of the bits q for i e {1, . . . , ?i + j — 1}. 

B. Generalized Tanner Graphs 

This work introduces the term generalized Tanner graph 
(GTG) to denote the Tanner grapld [8] corresponding to 

'The term Tanner graph has been used to describe different classes of 
graphical models by different authors. In this work, the term Tanner graph is 
used in its most restricted sense. That is to say, there is a bijection between 
the set of Tanner graphs for an [n, k, d] binary linear code C and the set of 
n — k X n parity check matrices for that code. 



the parity-check matrix of a generalized code extension. 
Specifically, let C be a degree-g generalized extension of the 
[n, k, d] binary linear block code C and let H ~ [hij] be an 
{n — k + g) X [n + g) parity-check matrix for C. The GTG 
associated with H is the bipartite graph = {U U W, £) 
with disjoint vertex classes: 



^ = K}?=i' and >V = {w,} 



n — k-\-g 
1 



(4) 



corresponding to the columns and rows of iJ, respectively. 
An edge connects Uj and Wi in Gj^ if and only if hij = 1. 
Note that a number of authors have previously considered such 
graphical representations of binary codes under the rubric of 
generalized parity-check matrices (cf. [9]). 

As an example, consider the [8, 4, 4] first-order Reed-Muller 
code C/jM(i,3)' any codeword of which can be decomposed via 
the squaring construction [7] as 

C = (Cl, C2, C3, C4, C5, Cg, C7, Cg) 

= (wi,U2, U3, U4, Ul + Vi,U2 + V2,U3 + V3, U4 + U4) (5) 

= (u,u + v) 

where u is drawn from the [4, 3, 2] single parity-check code 
and V is drawn from the [4, 1,4] repetition code. A degree-1 
generalized extension C^j/ji 3) can be formed by adding the 
partial parity bit C4 + cs — V4 to CjiM(i,3)- A parity-check 
matrix for Cra/(i,3) is 

"111100000" 
1 1 1 
1 1 1 
1 1 1 
1 1 1 



H 



RM{1,3) 



(6) 



Figure [T] illustrates the GTG corresponding to H 
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Fig. 1. Generalized Tanner graph for the [8,4,4] first-order Reed-Muller 
code implied by a squaring construction. Repetition and single parity-check 
constraints are labeled with '=' and '+' symbols, respectively. 



C. Conditionally Cycle-Free Generalized Tanner Graphs 

Hidden variable conditioning in graphical models is well- 
understood in the context of tail-biting trellises (cf. [10]). If 
a tail-biting trellis contains a hidden (state) variable, V, with 
alphabet size 2"' then optimal SISO decoding can be per- 
formed on this trellis by decoding on 2™ cycle-free conditional 
trellises (one per possible value of V) and then appropriately 
marginalizing over the results of the decoding rounds. This 
work applies hidden variable conditioning to generalized Tan- 
ner graphs in order to obtain cycle-free graphical models and 
thus optimal SISO decoding algorithms. 

As an example, consider again the GTG for CflM(i.3) 
illustrated in Figure [T] and suppose that the partial parity 
variable C4 + cg is not considered unknown but fixed to be 
either a or a 1. This conditional GTG is illustrated in Figure 
|2l Note that the conditional variable C4 + cs is now treated 
as a visible, deterministic variable as indicated by the half- 
edges labeled by '0/1' in Figure |2] The GTG illustrated in 
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Fig. 2. Conditionally cycle-free generalized Tanner graph for the [8, 4, 4] 
first-order Reed-Muller code. 



Figure |2] is cycle-free; optimal SISO decoding of a code with 
such a model can be accomplished by decoding twice on the 
conditionally cycle-free model (once with C4 + cs — and 
once with C4 + cg ~ 1) and then appropriately marginalizing 
over the results of the two decoding rounds [11]. 

More generally, a degree-f conditionally cycle-free general- 
ized Tanner graph contains / variables which can be fixed to 
remove all cycles. Optimal SISO decoding can be achieved 
by decoding 2^ times on a degree-/ conditionally cycle-free 
GTG (and appropriately marginalizing over the results). 

III. Conditionally Cycle-Free GTGs for 
First-Order Reed-Muller Codes 

Let Cflj\/(r.„i) denote the Reed-Muller code with parameters 



= 2'", k^Y. 



rf = 2" 



(7) 



Theorem 1: There exists a degree-(TO — 2) conditionally 
cycle-free generalized Tanner graph for CiiM{i.m)^ m > 3. 



Proof: By induction on to. The to = 3 case was 
shown by construction in Section |ll] Suppose that there exists 
a degree- (m — 3) conditionally cycle-free graphical model 
for Cjjj\/(i There exists a squaring construction for 

then 



C 



RM{l,m] 



such that if c e CflA/(i,„) 
c = (u, u + v) 



(8) 



where u e CflA/(i,m-i) and v e CRM(o,m-i) VI A GTG for 
this code is illustrated in Figure [3] The code vertex labeled 



C 



RM{l,m-l) 



corresponds to a GTG for that code. 
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Fig. 3. Generalized Tanner graph for the [2" 
code implied by a squaring construction. 
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By the induction hypothesis, a degree-(TO — 2) conditionally 
cycle-free GTG is formed by conditioning on the repetition 
constraint CjjM(o,m-i) and on to — 3 variables in the model 
corresponding to CijM(i,Tn-i)- □ 

It is well-known that a symbol-wise optimal SISO decoding 
algorithm for a code C implies a symbol-wise optimal SISO 
decoding algorithm for its dual [12], [13]. The condi- 
tionally cycle-free GTGs for first-order Reed-Muller codes 
described above thus imply optimal SISO decoding algo- 
rithms for the family of Reed-Muller codes with parameters 
[2™, 2™ — jTi — 1,4]: the extended Hamming codes. 

Let A„i and Cm denote the number of addition and com- 
parisor0 operations required by the optimal SISO decoding 
algorithm implied by the conditionally cycle-free GTG for 



C 



RM{1,' 



described above. Counting operations in the binary 



graphical model illustrated in Figure [3] yields the following 
recursion for A,„ and C„ 



3-2'" + 2A„_i, C„ 



2C„i_i. 



(9) 



Note that these formulae include the comparisons required 
for marginalizing over the results of the individual decoding 
rounds. It is clear that the complexity of SISO decoding using 
the proposed conditionally acyclic binary graphical models 

^Note that min-sum or min*-sum processing is assumed throughout this 
work so that Cm counts the number of min or min* operations. 



thus grows as O (rilogn). Ashikhim and Litsyn proposed a 
MAP decoding algorithm for the family of C^7v/(i.m) codes 
with similar asymptotic complexity based on fast Hadamard 
transforms [14]. Note that the complexity of bit-level trellis 
decoding of this code family grows as O (fi^)- 

In order to specifically demonstrate that conditionally cycle- 
free generalized Tanner graphs (CCF-GTGs) can imply sub- 
stantially less complex optimal SISO decoding algorithms 
than bit-level trellises, the number of addition and comparison 
operations for each SISO decoding algorithm is tabulated 
in Table U for a number of first-order Reed-Muller codes. 
The trellis complexity was evaluated by considering the total 
number of addition and comparison operations required to 
perform the full SISO BCJR algorithm [15] (rather that soft-in 
hard-out Viterbi decoding) on bit-level trellises with minimal 
state complexities as given by Lu and Huang in [16]. 





Trellis 


CCF-GTG 


Add 


Compare 


Add 


Compare 


CflM(l,3) 


176 


50 


72 


32 


C_RM(1,4) 


1008 


426 


192 


80 


C_RM(1,5) 


4720 


2202 


480 


192 


Crm(1,6) 


20336 


9850 


1152 


448 


Crm(ij) 


84336 


41530 


2688 


1024 


CflA/(l,8) 


343408 


170426 


6144 


2304 



TABLE I 

Complexity of trellis decoding vs. the proposed optimal SISO 

DECODING ALGORITHM. 

IV. Application: High-Rate SCCs 

In [3], Graell i Amat et al. studied a serially concatenated 
coding scheme consisting of a high-rate convolutional outer 
code and a recursive rate-1 inner code corresponding to a 
simplified discrete-time model of a precoded EPR4 magnetic 
recording channel. Specifically, the recursive channel model 
comprises a 1/1 precoder followed by a digital recording 
channel subject to intersymbol interference (ISI) with partial 
response polynomial 1 + D — — followed finally by an 
AWGN channel. Note that the precoder and ISI can be jointly 
decoded on an 8-state trellis. 

In this section, the performance of this scheme is compared 
to one which replaces the high-rate convolutional codes with 
extended Hamming codes. Specifically, 4 convolutional outer 
codes with input block size 4000 bits and respective rates 
8/9, 9/10, 11/12, and 16/17 are compared to four algebraic 
outer codes composed of the following mixtures of extended 
Hamming codes; 

• The mixture of 3 CjiM{3,5) and 69 Cj^a/(4,6) codewords 
resulting in a code with input block size 4011 bits and 
rate 4011/4512 = 0.8890 w 8/9. 

• The mixture of 56 Cjim{4,6) ™d 7 Cj^m{5J) codewords 
resulting in a code with input block size 4032 bits and 
rate 4032/4480 = 9/10. 

• The mixture of 30 C^MiiM) ™d 19 Ciim{5.7) codewords 
resulting in a code with input block size 3990 bits and 
rate 3990/4352 = 0.9168 « 11/12. 



. The mixture of 2 Crm (4.6), 26 Cb,m(5,7), and 3 Ci^A/(6,8) 
codewords resulting in a code with input block size 3975 
bits and rate 3975/4224 0.9411 w 16/17. 

As reported in [3], the SCCs using convolutional codes utilize 
s-random interleavers [17]. The codes using mixed extended 
Hamming outer codes utilize high sum-spread pseudo-random 
interleavers that were generated using the real-relaxation op- 
timization method described in [18]. 

Figure |4] compares the performance of the respective rate 
8/9 and 11/12 codes while Figure |5] compares the perfor- 
mance of the respective rate 10/11 and 16/17 codes. Note 
that the performance of the serially concatenated codes with 
convolutional outer codes is reported for 10 decoding iterations 
while the performance of the codes with mixed extended 
Hamming outer codes is reported for 10 and 20 decoding 
iterations. All curves correspond to min*-sum processing (or 
its dual-domain equivalent [13]). 
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Fig. 4. Bit error rate performance of the respective rate 8/9 and 11/12 
.serially concatenated codes. 

Observe in Figures |4] and |5] that the codes constructed 
with mixed extended Hamming outer codes compare favorably 
in terms of performance to those with convolutional outer 
codes of similar rates and input block sizes. The mixed 
extended Hamming codes compare favorably to their high-rate 
convolutional code counterparts in terms of complexity also. 
Table tabulates the average number of add and comparison 
operations per input bit, per decoding iteration, required for 
optimal SISO decoding (using the CCF-GTG algorithm) for 
each of the mixed extended Hamming codes. The rate k/k + 1 
convolutional outer-codes in [3] were decoded on the 16- (for 
fc = 8, 9, 10) and 32- (for k = 16) state trellises corresponding 
to their respective rate 1/1 + fc duals. Optimal SISO decoding 
on a rate 1/1 + fc, 16- (32-) state trellis requires at least 96 
(192) additions and 64 (128) comparisons per input bit, per 
decoding iteration. Thus, if one assumes that an addition and 
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Fig. 5. Bit error rate performance of the respective rate 9/10 and 16/17 
serially concatenated codes. 



comparison operation have roughly the same complexitjo then 
the complexity of the proposed mixed extended Hamming 
code SISO decoding algorithms are approximately 5 to 10 
times less than that of the respective high-rate convolutional 
code decoding algorithms proposed in [3]. 



Rate 


Add / Bit 


Compare / Bit 


8/9 


20.2 


7.9 


9/10 


20.7 


8.0 


11/12 


21.5 


8.2 


16/17 


22.8 


8.7 



TABLE II 

Average number of operations per decoding iteration, for 
optimal siso decoding of the mixed extended hamming codes. 

V. Conclusion 

Motivated by the search for optimal SISO decoding al- 
gorithms with complexity less than that of trellis decoding, 
this work studied conditionally cycle-free generalized Tanner 
graphs. It was shown that such models exist for the family 
of first-order Reed-Muller codes and that these models imply 
low-complexity SISO decoding algorithms for these codes and 
their duals: the extended Hamming codes. It was shown that 
the proposed low-complexity SISO decoding algorithms for 
the family of extended Hamming codes are particularly useful 
in the context of high-rate serially concatenated codes. 

A number of interesting directions for future work are mo- 
tivated by the present work. It would be interesting to search 
for more codes for which conditionally cycle-free generalized 
Tanner graphs offer complexity savings over trellis decoding. 

'Tliis assumption is clearly reasonable for min-sum processing. For min*- 
star processing, this assumption is also reasonable provided a table-lookup is 
used for the correction term in the min* operator. 



It would also be interesting to study the performance of the 
sub-optimal, iterative, SISO decoding algorithms implied by 
GTGs in which only a subset of hidden variables required 
to break all cycles are conditioned. Specifically, a study of 
codes with known algebraic decompositions may lead to 
novel low-complexity SISO decoding algorithms. There are 
also a number of interesting avenues of study concerning 
the application of the optimal SISO decoder for extended 
Hamming codes. Specifically, the algorithm presented in this 
work has a natural tree structure which can lead to particularly 
efficient implementations in hardware [19]. 
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